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ABSTRACT 


Recognition of artificial faces is an intriguing and testing problem and affects 
important applications in various regions, such as cooperation between 
human computers and data-oriented activity. Facial expression is the fastest 
correspondence methods for transmitting data. 

It is straightforward the outward appearance of an individual by looking at his 
/ her face yet somehow or other with regard to machines it ends up difficult to 
pass judgment on the outward appearance while using PC devices yet it is not 
incomprehensible in any way. This not only revealed any individual's 
affectability or sentiments, but it can also be used to make a judgement on the 
psychological views, yet again it could not fully understand the perception of 
human behaviour, the discovery of mental problems and fabricated human 
expressions. Expressions such as SAD, HAPPY, DISGUST, FEAR, ANGER, 
NEUTRAL and SURPRISE have been suggested in a broad range of processes 

This paper includes implementing face recognition along with facial 
expression recognition, analyzing recent and past research to extract effective 
and efficient methods for recognition of facial expression. 

KEYWORDS: Face Expression Recognition, Gabor Filter, Active Appearance Model 
(AAM), Flidden Markov Model (HMM) 

1. INTRODUCTION 

Face denotes a significant role in interaction, and it is also imperative to appear 
and perceive how an individual feels at a particular minute. Recognition of 
appearance externally is a strategy for perceiving expression. 


The face of a person has so many emotions that are 
recognized and understood by looking at the face. Happy, 
Fear, Sad, Disgust, Angry, Neutral, and Surprise might be 
these emotions and expressions. People have misconstrued 
once in a while that there is a stark difference between face 
recognition and facial expression. The important thing is as 
follows: 

Face Recognition: This application identifies or verifies an 
individual from a digital image or video. It includes 
information acquisition; processing of inputs, classification 
of face images and decision making. It is commonly used in 
voting verification, ATM banking, mobile password, and so 
on. 


Applications based on "Biometric Artificial Intelligence" can 
particularly differentiate a person by dissecting instances 
depending on the face surface and shape of the 
individual[3][4]. 

Facial recognition can be delegated recognition or holistic 
recognition where, together with a mixing unit, Principal 
Recognition involves outfit of highlight extractors or 
classifiers. 

Principal recognition or holistic recognition can be delegated 
where Principal Recognition involves a collection of 
highlighted extractors or classifiers together with a mixing 
unit. 


Facial Expression Recognition: 

This recognizes any person's facial expressions using either 
an image or a video clip or the person himself. It includes 
face detection, extraction of features, and clincludes 
classification of speech. It is commonly used in the 
healthcare, games and e-learning sectors. 

Woody Bledsoe[l], Helen Chan Wolf and Charles Bission[2] 
mainly used facial expression recognition technology. 
Together with Helen Chan and Charles Bission, Bledsoe 
chipped away during 1964 and 1965 using the PC to 
perceive human faces. 


Holistic Recognition-This provides the whole face a solo 
contribution to the structure of recognition. 

Recognition of facial expression is comprised as follows in a 
few significant steps:- 

1. Face detection and processing of image also known as 
Image Acquisition 

2. Pre-Processing 

3. Feature Extraction 

4. Expression Classification 
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Figurel. FER System Flow Chart 


Below is a brief introduction of the following steps involved 
in Facial Expression Recognition: 

(1) Image Acquisition or Face Detection: The image can be 
static image above all else, or successions of images must 
contain more information than a still static image. 

For Artificial Expression Recognition, 2D monochrome (dark 
scale) facial image successions are the most common type of 
images used. There are four methods that can efficiently 
distinguish a face, such as Learning Based, Feature Invariant, 
Template Matching, and Appearance Based. 



Figure2. Methods of face detection 



Figure3. Various Feature Extraction Techniques 


(2) Pre-Processing: It includes Signal Conditioning! such as 
elimination of noise, standardization against the variety of 
pixel position or splendor, etc.) The standardization of the 
image depends on eye or nostril references. 

(3) Feature Extraction: It shifts to a higher-level depiction 
over Pixel Data. It decreases the input space dimensionality. 
Using the feature extraction techniques, it tends to be 
completed. This includes a Discrete Cosine Transform[ DCT] 
Gabor Filter, Main Component Analysis [ PCA], Independent 
Component Analysis [ LDA]. 
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(4) Expression Classification: As examined before an 
person has so many expressions at certain time frames and a 
broad range of methods has been suggested to define those 
expressions. Recognizing expressions such as happy, sad, 
fear, disgust, angry, neutral, surprise. Facial expression 
recognition systems do not recognize either six expressions 
or the AUs more frequently than anything it requires. Wide- 
ranging study on facial expression analyses has been 
conducted over the past decades. The most commonly used 
facial expression analysis is performed as far as the action 
units suggested in the Facial Action Coding System are 
concerned and as far as all inclusive emotions are concerned: 
joy, sadness, anger, surprise, disgust and fear. The two main 
categories used in facial expression recognition are activity 
units (AUs) [10] and Ekman's prototypical facial 
expressions[ll]. 

2. FACIAL EXPRESSION RECOGNITION APPROACHES 
Facial Action Coding System (FACS) 

In 1978, the framework for estimating facial expressions was 
provided by Ekman et al.[12] called the FACS-Facial Action 
Coding System. FACS was developed by investigating the 
relationships between contraction of muscle(s) and changes 
in their face appearance. The Face can be divided into Upper 
Face and Lower Face Action units[13] and the resulting 
appearances are recognized as well. The figures show a part 
of the activity units that are joined. Muscle constraints 
responsible for a comparable activity are distinguished as an 
Action Unit (AU). The undertaking of expression examination 
using FACS is dependent on disintegrating observed 
expression into the action unit structure. There are 46 AUs 
that talk to outward appearance modifications and 12 AUs 
connected with the direction and direction of the eye stare. 
Activity units are deeply engaging as far as facial 
developments are concerned; in any event, they do not 
provide any information on the message to which they are 
speaking. 

Prototypical Facial Expression 

A usually small subset of seven important categories of 
expressions, observed through FER frameworks to be 
noticeable cross-sectionally over culture for use. As stated by 
the hypothesis of the Ekman[14], there are six vital 
expressions of emotions that are all inclusive to people from 
various nations and societies. They are anger, neutral, 
disgust, fear, happy, sad and surprise.. 

Rather than depicting the detailed facial features most facial 
expression recognition system attempts to perceive a small 
arrangement of prototypical passionate expressions. Some 
facial expressions are a mixture of more than one expression 
as for example of fear, sorrow and disgust, they state a 
combination that occurs. A few methodologies were used to 
overcome the above problem. There are two main classes of 
feature classification strategy: for instance, statistical non-AI 
approach, Euclidean and direct segregation research[15]. 
Machine learning approaches, for instance, Feed Forward 
Neural Network [16], Hidden Markov Model[17], Multilayer 
Perception, Support Vector Machine[18], and so on. 

There are two categories that can be divided into current 
methods: image-based strategies and model-based 
strategies. 

Picture-based methodologies that focus on perceiving facial 
operations by observing changes in the facial appearance of 
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the officer more frequently than doing whatever it takes not 
to autonomously and statically organize behavior or AUs 
More often than not, this kind of approach includes two main 
phases. To begin with, different facial highlights, e.g., optical 
stream [20] [21], unambiguous element estimation (e.g., 
wrinkle length and educational level] [22], Haar 
highlights[23], Local Binary Patterns (LBP] 
highlights [2 4] [25] autonomous part investigation (ICA)[26], 
include focuses[27], Gabor wavelets[28] and so on. The 
expressions/ AUs are acknowledged by recognition systems, 
such as Neural Networks, Support Vector Machines (SVM), 
rule-based methodology, AdaBoost classifiers, Sparse 
Representation [SR] classifiers, and so on, given the separate 
facial highlights. The ordinary shortcoming of picture-based 
approaches for AU recognition is that they will generally 
legitimately view each AU or certain AU mix individually and 
statically from the image data, regardless of the semantic 
and dynamic links between AUs, although some of them 
explore the transient characteristics of facial highlights. 

By using the links between AUs, model-based approaches 
overcome this deficiency and perceive the AUs at the same 
moment. Lien et al.[29] used many Hidden Markov Models 
(HMMs] to talk in time about the growth of facial operations. 
Classification is accomplished by selecting the AU or AU 
blend that amplifies the likelihood of the separate facial 
features generated by the HMM. Valstar et al.[30] used a 
mixture of SVMs and HMMs and flanked the SVM method for 
almost every AU by showing the temporary progress of facial 
activity. The two techniques misuse AU's worldly 
circumstances. As it may be, they suffer failure to exploit 
AU's spatial circumstances. The solution for this problem is: 
Tong and Ji used a Dynamic Bayesian scheme to show the 
spatiotemporal associations between AUs effectively and to 
achieve remarkable improvements over the picture-based 
method. In this paper, apart from showing the spatial and 
worldly connections between AUs, we also use the attitude 
and facial element focus information and, more critically, the 
coupling and associations between them. Expression 
Classification by classifiers should be feasible. It includes 
hidden Markov Model [HMM], Neural Network [NN], Support 
Vector Machine SVM, AdaBoost, Spare Representation [SRC]. 

3. COMPARATIVE ANALYSIS 

Similar investigation of the above mentioned facial 
expression recognition methodologies is shown in the table 
in this section. These methodologies are evaluated for 
standard facial expression databases such as Japanese 
woman outward appearance [JAFFE], FERET as far as the 
particular system's recognition rate, advantages and faults 
are concerned. 



Figure4. Graphical comparison of various facial expression 
techniques applied on standard face databases such as 
JAFFE, FERET etc. 


COMPARISON OF FACIAL EXPRESSION 
RECOGNITION METHODS 




AR-FACES FERET 


■ PCA ■ ICA ■ LBP bPCA+GABOR bLDP bLTP 
Figure5. Comparison of facial expression recognition 
methods 

4. FUTURE DIRECTIONS 

Facial expression recognition these days achieves a 
important place in various areas as it operates effectively 
under the circumstances that are required. A great deal of 
studies has been achieved and is going on in facial 
expression recognition, yet there is a need for progress, 
enhancement and development at the same moment. 
Considering the above examined outward appearance 
recognition techniques, which show better results in static 
conditions but failures under shifting conditions, e.g. change 
in modernity, variety present, maturing element and 
expressions. These are major causes that affect the display of 
almost all usual facial expression recognition techniques.. In 
this manner, future work should be feasible to overcome 
these problems and interpret the emotions of the individual 
"Environment in real-time under minimum constraints." For 
instance, enhancement and high objectives, there are two 
promising points for expression identification in the future. 
Both should be able to construct the recognition rate in 
recognition of facial expression. 


Tablet. Facial Expression Recognition approaches 
Comparison 


Recognition 

Approach 

Database 

Recognition 

Rate 

LBP 

JAFFE 

80% 

ICA 

FERET 

89% 

PCA 

AR-Faces 

70% 

PCA+ Gabor 

JAFFE 

85% 

PCA 

JAFFE 

70% 

LDP 

JAFFE 

89% 

LTP 

JAFFE 

89% 


5. CONCLUSION 

Recognition of facial expression is an exceedingly best 
assignment in the field of PC vision, which has achieved 
significance as a result of its various applications in the last 
few years. Many specialists have worked thoroughly to 
demonstrate that accurate recognition system for facial 
expression is mandatory. This paper offers guidance for 
different applications. For the growth and advancement of 
the new methodology, numerous experts have worked 
carefully to demonstrate accurate facial expression 
identification analysis. Key focuses, demerits and 
applications of the few techniques have been substantially 
evaluated in this paper. In addition, a close investigation was 
carried out to delineate the exhibition and accuracy of 
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various methodologies. This paper finally concludes by 
recommending to the scientist the possible instructions for 
enhancing the facial expression identification system 
exhibition. 
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